1. Field of the Invention
The present invention is a method and system for recognizing employees in a physical space based on automatic behavior analysis of the people in a preferred embodiment. The processes in the present invention are based on a novel usage of a plurality of computer vision technologies to analyze the behavior of the people from the plurality of input images.
2. Background of the Invention
The present invention is a method and system for recognizing employees in a physical space based on automatic behavior analysis of the people in a video, and the present invention can utilize any reliable automatic behavior analysis method and system in an exemplary embodiment. There have been earlier attempts for understanding people's behaviors, such as customers' shopping behaviors, captured in a video in a targeted environment, such as in a retail store, using cameras. However, the prior arts do not disclose all the features and novel usages of an automatic behavior analysis in the present invention, especially for employee recognition. The differences between the features that the present invention comprises in the automatic behavior analysis for employee recognition and the features in the relevant prior arts can be discussed as follows.
With regard to the temporal behavior, U.S. Pat. Appl. Pub. No. 2003/0002712 of Steenburgh, et al. (hereinafter Steenburgh) disclosed a method for measuring dwell time of an object, particularly a customer in a retail store, which enters and exits an environment, by tracking the object and matching the entry signature of the object to the exit signature of the object, in order to find out how long people spend in retail stores. However, Steenburgh is clearly foreign to the idea of using the dwell time of people in a physical space as one of the employee recognition criteria. Steenburgh do not distinguish employees from shoppers.
U.S. Pat. Appl. Pub. No. 2003/0058339 of Trajkovic, et al. (hereinafter Trajkovic) disclosed a method for detecting an event through repetitive patterns of human behavior. Trajkovic learned multidimensional feature data from the repetitive patterns of human behavior and computed a probability density function (PDF) from the data. Then, a method for the PDF analysis, such as Gaussian or clustering techniques, was used to identify the repetitive patterns of behavior and unusual behavior through the variance of the Gaussian distribution or cluster.
Although Trajkovic showed a method of modeling a repetitive behavior through the PDF analysis, Trajkovic is clearly foreign to the idea of recognizing employees in a physical space, and there is no explicit disclosure of applying their method for the employee recognition. Furthermore, Trajkovic is clearly foreign to the idea of non-repetitive employee behavior, such as non-repetitive employee trips to a predefined area in a physical space or employees' non-repetitive responsive behavior to a predefined event.
There have been earlier attempts for activity analysis in various other areas. The following prior arts disclosed methods for object activity modeling and analysis for the human body, using a video, in general.
U.S. Pat. Appl. Pub. No. 2003/0053659 of Pavlidis, et al. (hereinafter Pavlidis) disclosed a method for moving object assessment, including an object path of one or more moving objects in a search area, using a plurality of imaging devices and segmentation by background subtraction. In Pavlidis, the term “object” included customers, and Pavlidis also included itinerary statistics of customers in a department store. However, Pavlidis was primarily related to monitoring a search area for surveillance.
In addition, Pavlidis is clearly foreign to the idea of recognizing the employees as a distinction from the customers in a physical space. Therefore, Pavlidis does not explicitly discuss about how to distinguish between the moving objects as employees and the moving objects as customers. In the present invention, the behavior analysis for the employee recognition can work with context-dependent information. For example, even if the traffic pattern of the employees may look similar to that of customers, the present invention can differentiate the traffic pattern between the employees and customers according to context-dependent employee recognition criteria, such as spatial rules or predefined area rules. Pavlidis is clearly foreign to the idea of such rule application and employee recognition criteria.
U.S. Pat. Appl. Pub. No. 2004/0120581 of Ozer, et al. (hereinafter Ozer) disclosed a method for identifying activity of customers for a marketing purpose or activity of objects in a surveillance area, by comparing the detected objects with the graphs from a database. Ozer tracked the movement of different object parts and combined them to high-level activity semantics, using several Hidden Markov Models (HMMs) and a distance classifier.
Although Ozer also briefly mentioned tracking the activities of employees for human factors studies, Ozer is clearly foreign to the features of employee recognition in the present invention. First of all, while the present invention uses the tracking of the employees' trips in a physical space and the patterns in the tracking for the behavior analysis in the preferred embodiment, Ozer's approach is to model the object by invariant shape attributes and then compare the object model with a set of stored models. As a matter of fact, Ozer explicitly mentioned that their approach took a different method away from the motion detection and tracking in an argument, upon which the inventors in the present invention cannot agree completely, by the way. This clearly shows that Ozer is further foreign to the idea of employee recognition, utilizing the tracking and trip-based behavior analysis. Furthermore, Ozer is clearly foreign to the concept of employee recognition criteria that are disclosed in the present invention.
U.S. Pat. Appl. Pub. No. 2004/0131254 of Liang, et al. (hereinafter Liang) also disclosed the Hidden Markov Models (HMMs) as a way, along with the rule-based label analysis and the token parsing procedure, to characterize behavior. Liang disclosed a method for monitoring and classifying actions of various objects in a video, using background subtraction for object detection and tracking. However, Liang is particularly related to animal behavior in a lab for testing drugs, and Liang is clearly foreign to the concept and novel usage of employee recognition based on the behavior analysis applied by the employee recognition criteria in a physical space, such as a retail space.
Computer vision algorithms have been shown to be an effective means for detecting and tracking people. These algorithms have also been shown to be effective in analyzing the behavior of people in the view of the means for capturing images. This allows the possibility of connecting the visual information in the behavior analysis from the captured video images to the employee recognition. Any reliable automatic behavior analysis in the prior art may also be used for the behavior analysis part of the processes that will later be used as a basic tool for the employee recognition of the people in the exemplary embodiment of the present invention. However, the prior arts lack the employee recognition features that are disclosed in the present invention. The above prior arts are foreign to the concept of recognizing the employees based on automatic behavior analysis on video images and the employee recognition criteria applied to their behaviors, while tracking and analyzing the movement information of the employees in consideration of the employee recognition criteria in a physical space, such as a retail store.
Therefore, it is an objective of the present invention to provide a novel approach for recognizing the employees based on their behaviors, particularly utilizing the information from the automatic behavior analysis in video images that are captured by at least one means for capturing images in a physical space, and predefined employee recognition criteria.
It is another objective of the present invention to provide a novel solution for the employee recognition by automatically and unobtrusively analyzing the employees' behaviors based on the collection of trip information from the tracking of their trips without involving any hassle of requiring the employees to carry a cumbersome device in the automatic behavior analysis.